P values, confidence intervals, or confidence levels for hypotheses?

摘要

Null hypothesis significance tests and p values are widely used despite verystrong arguments against their use in many contexts. Confidence intervals areoften recommended as an alternative, but these do not achieve the objective ofassessing the credibility of a hypothesis, and the distinction betweenconfidence and probability is an unnecessary confusion. This paper proposes amore straightforward (probabilistic) definition of confidence, and suggests howthe idea can be applied to whatever hypotheses are of interest to researchers.The relative merits of the different approaches are discussed using a series ofillustrative examples: usually confidence based approaches seem moretransparent and useful, but there are some contexts in which p values may beappropriate. I also suggest some methods for converting results from one formatto another. (The attractiveness of the idea of confidence is demonstrated bythe widespread persistence of the completely incorrect idea that p=5% isequivalent to 95% confidence in the alternative hypothesis. In this paper Ishow how p values can be used to derive meaningful confidence statements, andthe assumptions underlying the derivation.) Key words: Confidence interval,Confidence level, Hypothesis testing, Null hypothesis significance tests, Pvalue, User friendliness.